Optimising ITS behaviour with Bayesian networks and decision theory

We propose and demonstrate a methodology for building tractable normative intelligent tutoring systems (ITSs). A normative ITS uses a Bayesian network for long-term student modelling and decision theory to select the next tutorial action. Because normative theories are a general framework for rational behaviour, they can be used to both define and apply learning theories in a rational, and therefore optimal, way. This contrasts to the more traditional approach of using an ad-hoc scheme to implement the learning theory. A key step of the methodology is the induction and the continual adaptation of the Bayesian network student model from student performance data, a step that is distinct from other recent Bayesian net approaches in which the network structure and probabilities are either chosen beforehand by an expert, or by efficiency considerations. The methodology is demonstrated by a description and evaluation of CAPIT, a normative constraint-based tutor for English capitalisation and punctuation. Our evaluation results show that a class using the full normative version of CAPIT learned the domain rules at a faster rate than the class that used a non-normative version of the same system

Large-Scale Deployment of Three Intelligent Web-based Database Tutors

We present our experiences with DatabasePlace, a Web portal aimed at university-level students enrolled in database courses. The portal was established by Addison-Wesley in January 2003. Besides presenting information about the textbooks, the portal also provides additional domain information, online quizzes and three Intelligent Tutoring Systems developed by the Intelligent Computer Tutoring Group (ICTG). We briefly present the three systems, and then discuss our experiences. We also compare the DatabasePlace students to our local students using the three ITSs

Eye Tracking and Studying Examples: How Novices and Advanced Learners Study SQL Examples

Eye tracking provides information about a user’s eye gaze movements. For many years, eye tracking has been used in Human Computer Interaction (HCI) research. Similarly, research on computerised educational systems also relies heavily on students’ interactions with systems, and therefore eye tracking has been used to study and improve learning. We have recently conducted several studies on using worked examples in addition to scaffolded problem solving. The goal of the project reported in this paper was to investigate how novices and advanced students learn from examples. The study was performed in the context of SQL-Tutor, a mature Intelligent Tutoring System (ITS) that teaches SQL. We propose a new technique to analyse eye-gaze patterns named EGPA. In order to comprehend an SQL example, students require the information about tables’ names and their attributes which are available in a database schema. Thus, if students paid attention to the database schema, they would understand SQL examples more easily. We analysed students’ eye movement data from different perspectives, and found that advanced students paid more attention to database schema than novices. In future work, we will use the findings from this study to provide proactive feedback or individualised amounts of information

Investigating Engagement and Learning Differences between Native and EFL students in Active Video Watching

Video-based learning (VBL) requires good listening and reading comprehension skills, which could be challenging for English as a foreign language (EFL) students. In this paper, we investigate the differences between EFL and Native English speakers in a VBL platform called AVW-Space, in order to identify potential interventions that would be helpful for EFL students. AVW-Space provides note-taking, peer-reviewing, visualisations and personalised nudges to support engagement in VBL. Although previous studies on AVW-Space showed these supports were effective for increasing engagement, we discovered significant differences in learning outcomes and engagement between EFL/Native students, which stem from different learning strategies, background knowledge and language barriers. This research contributes to using learning analytics to understand better the differences between EFL and Native students, and providing more specialised support for EFL students in VBL

How much support is necessary for self-regulated learning?

Self-regulated learning is crucial for learning success, and is even of greater importance for online learning as there is less support and feedback available to students. We describe a simple intervention designed to support self-regulated learning in the context of SQL-Tutor, a mature intelligent tutoring system. SQL-Tutor logged data about all interactions students performed, including interactions with the SRL support. Frequency-based analyses did not identify any differences in behaviors of low or high scoring students. However, epistemic network analysis identified significant differences in how students use help available from SQL-Tutor. Students who scored low on the SQL test asked for high-level help (in the form of partial or full solution), copied the provided solutions and submitted them as their own. We conclude that additional support is necessary for students with weak self-regulation skills

Crowdsourcing Content Creation for SQL Practice

Crowdsourcing refers to the act of using the crowd to create content or to collect feedback on some particular tasks or ideas. Within computer science education, crowdsourcing has been used -- for example -- to create rehearsal questions and programming assignments. As a part of their computer science education, students often learn relational databases as well as working with the databases using SQL statements. In this article, we describe a system for practicing SQL statements. The system uses teacher-provided topics and assignments, augmented with crowdsourced assignments and reviews. We study how students use the system, what sort of feedback students provide to the teacher-generated and crowdsourced assignments, and how practice affects the feedback. Our results suggest that students rate assignments highly, and there are only minor differences between assignments generated by students and assignments generated by the instructor.Peer reviewe

Supporting Novices and Advanced Students in Acquiring Multiple Coding Skills

We present our study on PyKinetic with various activities to target several skills: code tracing, debugging, and code writing. Half of the participants (control group) received the problems in a fixed order, while for the other half (experimental group) problems were selected adaptively, based on their performance. In a previous paper, we discussed the general findings from the study. In this paper we present further analyses and focus on differences between low performing students and students with higher pre-existing knowledge. We hypothesized that: (H1) novices will benefit more than advanced students, and (H2) advanced students in the experimental group will benefit more than those in the control group. The results confirmed H1 and revealed that this version of PyKinetic was more beneficial for novice learners. Moreover, novices showed evidence of learning multiple skills: code writing, debugging and code tracing. However, we did not have enough evidence for hypothesis H2

Investigating the effects of learning activities in a mobile Python tutor for targeting multiple coding skills.

Mobile devices are increasingly being utilized for learning due to their unique features including portability for providing ubiquitous experiences. In this paper, we present PyKinetic, a mobile tutor we developed for Python programming, aimed to serve as a supplement to traditional courses. The overarching goal of our work is to design coding activities that maximize learning. As we work towards our goal, we first focus on the learning effectiveness of the activities within PyKinetic, rather than evaluating the effectiveness of PyKinetic as a supplement resource for an introductory programming course. The version of PyKinetic (PyKinetic_DbgOut) used in the study contains five types of learning activities aimed at supporting debugging, code-tracing, and code writing skills. We evaluated PyKinetic in a controlled lab study with quantitative and qualitative results to address the following research questions: (R1) Is the combination of coding activities effective for learning programming? (R2) How do the activities affect the skills of students with lower prior knowledge (novices) compared to those who had higher prior knowledge (advanced)? (R3) How can we improve the usability of PyKinetic? Results revealed that PyKinetic_DbgOut was more beneficial for advanced students. Furthermore, we found how coding skills are interrelated differently for novices compared to advanced learners. Lastly, we acquired sufficient feedback from the participants to improve the tutor

Evaluation of Influence Factors on the Visual Inspection Performance of Aircraft Engine Blades

Background—There are various influence factors that affect visual inspection of aircraft engine blades including type of inspection, defect type, severity level, blade perspective and background colour. The effect of those factors on the inspection performance was assessed. Method—The inspection accuracy of fifty industry practitioners was measured for 137 blade images, leading to N = 6850 observations. The data were statistically analysed to identify the significant factors. Subsequent evaluation of the eye tracking data provided additional insights into the inspection process. Results—Inspection accuracies in borescope inspections were significantly lower compared to piece-part inspection at 63.8% and 82.6%, respectively. Airfoil dents (19.0%), cracks (11.0%), and blockage (8.0%) were the most difficult defects to detect, while nicks (100.0%), tears (95.5%), and tip curls (89.0%) had the highest detection rates. The classification accuracy was lowest for airfoil dents (5.3%), burns (38.4%), and tears (44.9%), while coating loss (98.1%), nicks (90.0%), and blockage (87.5%) were most accurately classified. Defects of severity level S1 (72.0%) were more difficult to detect than increased severity levels S2 (92.8%) and S3 (99.0%). Moreover, visual perspectives perpendicular to the airfoil led to better inspection rates (up to 87.5%) than edge perspectives (51.0% to 66.5%). Background colour was not a significant factor. The eye tracking results of novices showed an unstructured search path, characterised by numerous fixations, leading to longer inspection times. Experts in contrast applied a systematic search strategy with focus on the edges, and showed a better defect discrimination ability. This observation was consistent across all stimuli, thus independent of the influence factors. Conclusions—Eye tracking identified the challenges of the inspection process and errors made. A revised inspection framework was proposed based on insights gained, and support the idea of an underlying mental model

Improving Face-to-Face Communication Skills using Active Video Watching

Although communication skills are widely recognized as crucial for effective software development teams, many graduates lack such skills, which are difficult to teach. We adopt the active video watching (AVW) approach to teach face-to-face communication skills to second-year software engineering project course. We conducted an experiment with AVW-Space, an online platform which supports video-based learning. The participants watched and commented on tutorial videos first, and later on provided videos of meetings. In the last phase, the participants commented on the recording of their own team meeting. We found that students who commented and rated others’ comments increased their conceptual understanding of face-to-face communication skills